Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Lambda support in VM: Wolfgang's PR plus some simplifications (removing mask, clarifying ops) #15387

Open
wants to merge 3 commits into
base: 11-21-lambda-compiler-completion
Choose a base branch
from

Conversation

brmataptos
Copy link
Contributor

@brmataptos brmataptos commented Nov 25, 2024

Description

This PR started with a cherry-pick commit of Wolfgang's draft PR 15171 = commit #d85b919.

On top of that, I

  • Remove the Mask feature since it complicates review,
  • Extend and rename the opcodes to make them more general and clear
    • Any function value can be have arguments bound to it in advance,
      not requiring a defined function handle except in LdFunction[Generic].
  • Tried to avoid unnecessary changes, but may have accidentally done so.
    • It took me a while to realize that we do indeed need a type parameter to
      EarlyBind and Invoke because of the way the verifier works today.
  • add some semantics comment to file_format.rs which I'd worked out previously.

The current operations are:
- LdFunction, LdFunctionGeneric = generate a closure from a defined function
- EarlyBind = bind more arguments to any closure
- Invoke = call a closure with final arguments

As noted by Wolfgang in his commit, we are still lacking:

  • function value representation and interpreter
  • function value serialization and deserialization
  • some connections with front-end

How Has This Been Tested?

Barely tested, just running through existing tests

Key Areas to Review

Did I lose any verifier support when modfiying Wolfgang's opcodes?

Copy link

trunk-io bot commented Nov 25, 2024

⏱️ 1h 30m total CI duration on this PR
Job Cumulative Duration Recent Runs
rust-move-tests 13m 🟩
rust-move-tests 12m 🟩
rust-move-tests 12m 🟩
rust-move-tests 12m 🟥
rust-move-tests 12m 🟥
rust-cargo-deny 10m 🟩🟩🟩🟩🟩 (+1 more)
check-dynamic-deps 9m 🟩🟩🟩🟩🟩 (+1 more)
rust-move-tests 4m 🟥
general-lints 3m 🟩🟩🟩🟩🟩 (+1 more)
semgrep/ci 2m 🟩🟩🟩🟩🟩 (+1 more)
file_change_determinator 1m 🟩🟩🟩🟩🟩 (+1 more)
permission-check 21s 🟩🟩🟩🟩🟩 (+1 more)
permission-check 16s 🟩🟩🟩🟩🟩 (+1 more)
check-branch-prefix 1s 🟩

🚨 1 job on the last run was significantly faster/slower than expected

Job Duration vs 7d avg Delta
check-dynamic-deps 5m 1m +236%

settingsfeedbackdocs ⋅ learn more about trunk.io

@brmataptos brmataptos changed the title Verbatim cherry-pick of Wolfgang's draft PR 15171 = commit #d85b919 Wolfgang's VM closure support + reconciliation with lambda simplifications (removing mask, clarifying ops) Nov 25, 2024
@brmataptos brmataptos changed the title Wolfgang's VM closure support + reconciliation with lambda simplifications (removing mask, clarifying ops) Lambda support in VM: Wolfgang's PR plus some simplifications (removing mask, clarifying ops) Nov 25, 2024
@brmataptos brmataptos marked this pull request as ready for review November 25, 2024 05:07
// Since bytecode version 8
LdFunction(_) => Opcodes::LD_FUNCTION,
LdFunctionGeneric(_) => Opcodes::LD_FUNCTION_GENERIC,
Invoke(_) => Opcodes::INVOKE,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Probably we want InvokeFunction, EarlyBindFunction for consistency?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

Though I wonder if we should rename Call as CallFunctionHandle for similar consistency. :-)

]
}

pub static INITIAL_COST_SCHEDULE: Lazy<CostTable> = Lazy::new(|| {
let mut instrs = bytecode_instruction_costs();
// Note that the DiemVM is expecting the table sorted by instruction order.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you give more info on why this is removed?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note that new_from_instructions sorts the parameter the same way, so no need to sort again here.

wrwg and others added 3 commits November 30, 2024 22:46
[move-vm] Types and opcodes for closures

This PR implements the extensions needed in the file format for representing closures:

- The type (SignatureToken) has a new variant `Function(args, result, abilities)` to represent a function type
- The opcodes are extendeed by operations `ClosPack`, `ClosPackGeneric`, and `ClosEval`

This supports bit masks for specifyinng which arguments of a function are captured when creating a closure.

Bytecode verification is extended to support the new types and opcodes. The implementation of consistency, type, and reference safety should be complete. What is missing are:

- File format serialization
- Interpreter and value representation
- Value serialization
- Connection of the new types with the various other type representations
…rser work:

- `LdFunction`, `LdFunctionGeneric` = generate a closure from a defined function
- `EarlyBind` = bind more arguments to any closure
- `Invoke` = call a closure with final arguments
Add some description/semantics to `file_format.rs` to better describe
the implementation which will be needed.

Get rid of complex Mask calculations in favor of simpler early bninding of
`k` initial arguments.

Hopefully keep all bytecode verifier operations working the same.
@brmataptos brmataptos force-pushed the 11-21-lambda-compiler-completion branch from ae4444b to 8b37927 Compare December 1, 2024 06:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants